Adaptive field/frame filter for interlaced video signals

ABSTRACT

A motion adaptive vertical filter is used to filter an interlaced video signal on a frame basis in areas of the image that are not in motion and on a field basis in areas that are in motion. A soft switch mixes the field filtered data and the frame filtered data in areas where there is relatively little motion to prevent artifacts which may be caused by abruptly switching between the two vertical filtering schemes. The frame vertical filter reduces the vertical resolution of still NTSC images to 180 cycles per picture height (CPH) while the field filter reduces the vertical resolution to 90 CPH.

BACKGROUND OF THE INVENTION

The present invention concerns spatial filtering of video images and, inparticular, an adaptive filtering system which switches betweenfield-based and frame-based filters on a block-by-block basis responsiveto motion in the corresponding blocks of the video image.

It is often desirable to filter interlaced video signals for manydifferent applications. For example, if the image is to be reduced insize, it is desirable to low-pass filter the image, both horizontallyand vertically, to prevent aliasing distortion in the subsampled image.For image data compression applications it may also be desirable toband-limit the video signal prior to compression to reduce thevisibility of artifacts arising from the image compression process.

A purely horizontal filter is fairly simple to implement since itinvolves temporally filtering successive picture elements (pixels) oneach line of the interlaced video signal. Vertical filtering of an imagesignal, however, is more complex since there are problems implementing avertical filter, either on a purely frame basis or a field basis.

To understand these problems, it is helpful to describe the structureand timing of an interlaced video signal. FIG. 1a illustrates an imageframe of, for example, an NTSC video signal. The exemplary frameexhibits the highest vertical resolution that may be obtained in such asignal. This is achieved by having alternate lines, A, of the image at adifferent signal level than the intervening lines, B. Since there are480 lines in the active portion of the NTSC video signal, the maximumvertical resolution that can be achieved is 240 cycles per pictureheight (CPH) as dictated by the Nyquist criteria.

A typical NTSC video signal, however, is neither transmitted, receivednor displayed as shown in FIG. 1a. Instead, the signal is transmitted astwo fields shown in FIGS. 1b and 1c respectively. As shown in FIG. 1b,all of the lines of the first field have the level A while, as shown inFIG. 1c, all of the lines of the second field have the level B. Due topersistence of vision in the human eye, the two fields, having a displayrate of 60 fields per second, are integrated into the frame image shownin FIG. 1a, having a repetition rate of thirty frames per second.

If there is no motion in the image represented by the video signal, thatis to say, if the pixels in the image are the same from frame to frame,it is preferable to filter the image as a frame instead of as twofields. This is illustrated by the frame and fields shown in FIGS. 1a,1b and 1c. If the image frame shown in FIG. 1a were vertically filteredby a filter having a cut-off resolution of, for example, 180 CPH, theresult would be a frame image having lower resolution than the originalframe image. If, however, the fields shown in FIGS. 1b and 1c werepassed through the same filter, there would be no difference between theimage produced by the input signal and that produced by the outputsignal. This can be seen because there is no high-resolution componentin either of the two fields while there is a high-resolution componentin the single frame which results from combining the two fields.

On a more theoretical level, FIG. 2a shows the passband/stopbandspectrum of a frame vertical filter for the case where the cut-offresolution is 180 CPH. As can be seen in FIG. 2a, the shaded region,representing vertical frequencies passed by the filter, is limitedbetween 0 and 180 CPH. Image components having vertical resolutionbetween 180 CPH and 240 CPH are not passed by this vertical filter. FIG.2b shows this same filter applied on a field basis. As can be seen, theintegrated frame image generated from this filtered field signal has astopband between 90 and 150 CPH and a passband elsewhere. This isobviously not a low-pass filtered characteristic.

On moving images, field filtering may still be more desirable than framefiltering because it substantially reduces temporal artifacts which maybe caused by frame filtering.

For example, FIG. 3 illustrates how the image of a box moving to theright would be represented by an interlaced video signal. The outline ofthe box in the first field is illustrated by the solid line and, in thesecond field, by the broken line. When fields one and two are combinedto form a frame, regions of serrations (i.e. spurious alternate lines)appear along the vertical edges of the box. These serrations occupy ahorizontal dimension equal to the motion of the box in one field period.If these image fields were to be combined into a single frame, framefiltered and then displayed, these artifacts would be displayed, albeitat a reduced resolution in each field of the frame.

If, however, the image were processed as two successive fields, therewould be no temporal artifacts, each field image would have the objectin its proper position in the time sequence. Obviously, such temporalartifacts are to be avoided; therefore, in moving areas of an image,field filtering may be preferred to frame filtering.

SUMMARY OF THE INVENTION

The present invention is embodied in an adaptive field/frame filteringsystem. The system includes a motion detector which determines therelative motion between two fields of each frame, a frame-orientedfilter and a field-oriented filter. The motion detector is coupled tocircuitry which blends the signals provided by the frame filter andfield filter on a block-by-block basis so that a stationary image isentirely frame-filtered, a fast-moving image is entirely field-filtered,and images exhibiting other degrees of motion include somefield-filtered and some frame-filtered components.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1a, 1b and 1c (prior art) are frame and field diagrams useful fordescribing the environment of the present invention.

FIGS. 2a and 2b are graphs of vertical resolution versus horizontalfrequency which are useful for describing the operation of respectiveframe and field-based low-pass filters.

FIG. 3 (prior art) is a diagram of an interlaced video signal which isuseful for describing temporal artifacts.

FIG. 4 is a block diagram of an exemplary field/frame-based filtersystem according to the present invention.

FIG. 5 is a pixel diagram which illustrates the operation of the frameconverter and block converter shown in FIG. 4.

FIG. 6 is a graph of scaling factor versus summed squared differencewhich is useful for explaining the blending of the field andframe-filtered signals produced by the circuitry shown in FIG. 4.

FIG. 7 is a graph of signal amplitude versus normalized relativefrequency which is useful for describing the frequency responsecharacteristic of the field and frame filters shown in FIG. 4.

DETAILED DESCRIPTION

FIG. 4 is a block diagram of an adaptive field/frame filter system inaccordance with the present invention. In FIG. 4, an interlaced videosignal provided, for example, by a conventional video camera is appliedin parallel to three circuit portions. The first circuit portion, adecision path 410, locates blocks in the image which exhibit motion fromframe to frame. A second circuit portion is a frame filter path 420which is used to low-pass filter areas of the image which are found tobe substantially free of motion. The remaining circuit portion is afield filter path 430 which filters the remaining areas of the image,that is to say, those portions of the image which exhibit motion fromframe to frame. The output signals of the frame filter path and thefield filter path are combined by summing circuitry 440 to generate ablocked video output signal. This signal may also be applied to a rasterconverter 442, shown in phantom, to produce an interlaced output videosignal.

The decision path 410 includes a frame converter 412 which combinessuccessive fields of the received interlaced video signal to producesuccessive frames of video signal information. The frame converter 412may include, for example, two frame memories (not shown) arranged in aping-pong configuration such that while one frame memory is receivingpairs of interlaced video fields and forming them into frames, the otherframe memory is providing pixels from a previously stored frame to theframe delay 414 and block converter 416 of the decision path 410 and tothe frame filter 422 of the frame filter path, described below.

The frame delay 414 delays the pixel samples provided by the frameconverter 412 by one frame interval. The output of the frame delay 414is applied to a block converter 415. Block converters 415 and 416 readpixel values, sixty-four at a time, from the respective frame delay 414and frame converter 412.

An exemplary block of pixels is shown in FIG. 5. As can be seen fromFIG. 5, the block includes eight adjacent pixels from each of eightsuccessive lines in the frame. These lines include four lines from fieldinterleaved with four lines from field two. The blocks of pixelsprovided by the block converter 415 represent image information from aprevious frame, while the blocks provided by the block converter 416represent image information from the present frame.

The motion detector 418 calculates a measure of the motion in theunderlying image between the previous frame and present frame as the sumof squared differences (SSD) between the present frame and the previousframe. The SSD value for pixels a_(ij) from the present frame and b_(ij)from the previous frame is calculated according to equation (1).##EQU1##

Without loss of generality, if we assume that pixel values can rangebetween a value of zero for black and one for white, then the SSD valuescan range between zero and one. If the video signal contains no noiseand there was no motion between the images from frame to frame, the SSDvalue would be equal to zero. Higher levels of motion, such that allpixels in the current field are white and all pixels in the previousfield are black, would produce an SSD value of one. Pixel values in thetwo frames between these extremes will produce SSD values between zeroand one.

In the present embodiment of the invention, the SSD value is used togenerate a soft switching value, α, which is, in turn, used to switchthe system between frame filtering and field filtering the interlacedvideo signals. The value α is generated by applying the SSD valuescalculated according to equation (1) to a read only memory (ROM) whichis programmed as a look-up table (LUT). An exemplary program for the LUTis shown in FIG. 6. In this Figure, the value of α is unity for SSDvalues between zero and 0.15. For SSD values between 0.15 and 0.45, thevalue of α decreases from unity to zero with a slope of -2.5. For SSDvalues greater than 0.45, the value of α is zero.

As described above, the frame filter path 420 receives the output signalfrom frame converter 412 into a frame filter 422. Frame filter 422 maybe, for example, a twenty-one tap finite impulse response (FIR) filterhaving the weighting coefficients listed in Table I.

                  TABLE I                                                         ______________________________________                                                H[-10] = H[10] = -0.002553                                                    H[-9] = H[9] = 0.002570                                                       H[-8] = H[8] = 0.0                                                            H[-7] = H[7] = 0.008690                                                       H[-6] = H[6] = 0.021158                                                       H[-5] = H[5] = -0.024368                                                      H[-4] = H[4] = 0.0                                                            H[-3] = H[3] = 0.060948                                                       H[-2] = H[2] = -0.145527                                                      H[-1] = H[1] = 0.220548                                                       H[0]= 0.751827                                                        ______________________________________                                    

In the filter, each tap is separated from the next successive tap by aone-horizontal-line period (1H) ms delay. The output signal of each tapin the 20H delay line is weighted according to the coefficients in TableI and then summed to produce an output value. Of course, the output tapsH[-8], H[-4], H[4]and H[8], which have weights of zero may be omittedfrom the weighting and summing operations.

FIG. 7 is a graph of amplitude versus normalized relative frequencywhich illustrates the frequency-response characteristic of the framefilter 422. The horizontal axis in this figure is unitless andrepresents normalized frequencies, that is to say, frequency valueswhich have been divided by a normalizing frequency value. As shown inFIG. 7, this filter has a 6 dB point at a normalized relative frequencyof 0.75. When used as a frame filter, the relative frequency 1corresponds to 240 CPH; thus, in this instance, the normalized relativefrequency 0.75 corresponds to 180 CPH.

Referring to FIG. 4, the output signal of the frame filter 422 isapplied to a block converter 424 which operates in substantially thesame manner as the block converters 415 and 416 described above. Blockconverter 424, in turn, provides blocks, each containing 64frame-filtered video pixels, to a scaling circuit 426. The scalingcircuit 426 is coupled to receive the signal α provided by the motiondetector 418, as described above. Scaling circuit 426 thus multipliesthe pixel values provided by block converter 424 by the factor α andapplies the result to one input port of summing circuit 440. The otherinput port of summing circuit 440 is coupled to receive correspondingblocks of pixels which have been field filtered, combined into a frameformat and converted to block format. These pixels are provided throughthe field filter path 430.

In the field filter path 430, the input interlaced video signals arereceived directly by a field filter 432. In the exemplary embodiment ofthe invention, the field filter 432 may be identical in structure to theframe filter 422. Since, however, the input signal to the field filter432 is an interlaced video signal, the field filter 432 operates ontwenty-one successive lines from a single field. This normalizingfrequency and, so, the interpretation of the frequency-responsecharacteristic shown in FIG. 7. For a field filter, a normalizedrelative frequency of 1 corresponds to 120 CPH; thus, the breakpoint atthe normalized relative frequency of 0.75 now corresponds to 90 CPH.

The filtered pixel values provided by the field filter 432 are appliedto a frame converter 434 which may be substantially the same as theframe converter 412 described above. Converter 434 provides frames ofvideo pixels to a block converter 436 which may operate in the samemanner as the block converters 415, 416 and 424 described above. The 64pixel blocks of field filtered video signals provided by the blockconverter 436 are applied to a scaling circuit 438 which is also coupledto the motion detector 418. Instead of scaling each pixel value in theblock by α as in scaling circuit 426, however, scaling circuit 438scales each pixel value in the block by a factor of 1-α. The outputsignal of the scaling circuit 438 is the signal which is applied to thesecond input port of the summing circuit 440.

As described above, the value α is related to the SSD as calculated byequation (1) in the manner shown in FIG. 6. Thus, when the measure ofmotion between the previous frame and the present frame is small, suchthat the SSD value calculated by the motion detector 418 is less than0.15, the summing circuit 440 provides a pure frame-filtered signal asits output. As the measure of motion increases above this value, acombination of frame-filtered and field-filtered signals are provided.When the motion between the previous frame and the present frame isabove a threshold value, an SSD value of 0.45, the output signal of thesumming circuit 440 is blocks of pixels in frame order which representtwo fields, where each of the two fields has been separatelyfield-filtered.

This combination of field-filtered signals and frame filtered signalsallows a stationary image, in which high vertical resolution isdesirable, to be displayed using entirely frame-filtered signals. A fastmoving image, however, in which temporal resolution is more important isdisplayed using entirely field-filtered signals. Images exhibiting otherdegrees of motion include some field-filtered and some frame filteredcomponents to compromise between vertical and temporal resolution.

As shown in FIG. 4, the output signal of the summing circuit 440 may beapplied as a blocked video signal to a compression system or,alternatively, it may be applied to a raster converter 422, shown inphantom, to produce an interlaced video output signal.

Although the frame and field filters in the exemplary embodiment of theinvention have been shown as being the same, it is contemplated thatdifferent frame and field filters may be used with correspondingresults. Furthermore, it is contemplated that the frame and fieldfilters 422 and 432 may have different types of frequency-responsecharacteristics, for example, high-pass, bandpass, or notch.

Although the invention has been described in terms of an exemplaryembodiment, it is contemplated that it may be practiced as outlinedabove within the spirit and scope of the appended claims.

The invention claimed is:
 1. A motion adaptive video signal filteringsystem suitable for filtering an interlaced video signal in which aframe of video information is represented by first and second interlacedfields and each of said fields includes a plurality of lines of pixelvalues, said system comprising:field filtering means for processingcorresponding ones of said pixel values on each of a plurality ofsuccessive lines of one of said fields to generate a field filteredvideo signal; frame conversion means for combining successive pairs ofsaid first and second interlaced fields to generate successive frames ofsaid video signal; frame filtering means for processing correspondingones of said pixel values on each of a plurality of successive lines ofone of the frames to generate a frame filtered video signal; motiondetection means, responsive to differences in successive ones of theframes provided by the frame conversion means, for selectively combiningthe field filtered video signal and the frame filtered video signal togenerate an output signal.
 2. A motion adaptive video signal filteringsystem according to claim 1, wherein the motion detection meansincludes:first block conversion means for converting the pixel values ina first one of said successive frames into blocks of pixel values whereeach block includes N times M pixel values, representing imageinformation conveyed by N successive pixel values in each of Msuccessive lines, where N and M are integers; second block conversionmeans for converting the pixel values in a second one of said successiveframes into blocks of pixel values where each block includes N times Mpixel values; comparison means for determining respective differences inmagnitude between each of the blocks provided by the first blockconversion means and a respective block provided by the second blockconversion means; and means, responsive to the determined difference,for providing only the frame filtered video signal when the differenceis less than a first predetermined value and for providing only thefield filtered video signal when the difference is greater than a secondpredetermined value.
 3. A motion adaptive video signal filtering systemaccording to claim 2, wherein the motion detection means furtherincludes means, responsive to the determined difference for additivelycombining the field filtered video signal and the frame filtered videosignal when the difference is greater than the first predetermined valueand less than the second predetermined value.
 4. A motion adaptive videosignal filtering system according to claim 3, wherein:the framefiltering means includes a finite impulse response (FIR) low-pass filterwhich attenuates vertical spatial frequency components of each framehaving resolutions greater than 180 characters per picture height (CPH)relative to other vertical spatial frequency components; and the fieldfiltering means includes an FIR low-pass filter which attenuatesvertical spatial frequency components of each field having resolutionsgreater than 90 CPH relative to other vertical spatial frequencycomponents.
 5. A motion adaptive video signal filtering system accordingto claim 4, wherein the comparison means includes means for determiningthe average value of the squares of the differences between each pair ofcorresponding pixel values in the blocks provided by the respectivefirst and second block conversion means.
 6. A method ofmotion-adaptively filtering an interlaced video signal in which a frameof video information is represented by first and second interlacedfields and each of said fields includes a plurality of lines of pixelvalues, said method comprising the steps of:A) processing correspondingones of said pixel values on each of a plurality of successive lines ofone of said fields to generate a field filtered video signal; B)combining successive pairs of said first and second interlaced fields togenerate successive frames of said video signal; C) processingcorresponding ones of said pixel values on each of a plurality ofsuccessive lines of one of the frames to generate a frame filtered videosignal; D) calculating a measure of difference in magnitude of pixelvalues in corresponding areas of successive ones of the frames generatedat step B); and E) selectively combining the field filtered video signaland the frame filtered video signal, response to the measured differencein magnitude, to generate an output signal.
 7. A method according toclaim 6, wherein the step D) includes the steps of:D1) converting thepixel values in a first one of said successive frames into blocks ofpixel values where each block includes N times M pixel values,representing image information conveyed by N successive pixel values ineach of M successive lines, where N and M are integers; D2) convertingthe pixel values in a second one of said successive frames into blocksof pixel values where each block includes N times M pixel values; D3)determining respective differences in magnitude between each of theblocks provided by the first block conversion means and a respectiveblock provided by the second block conversion means; and D4) providingonly the frame filtered video signal when the determined difference isless than a first predetermined value, providing only the field filteredvideo signal when the determined difference is greater than a secondpredetermined value, and providing an additive combination of the framefiltered video signal and the field filtered video signal when thedetermined difference is greater than the first predetermined value andless than the second predetermined value.
 8. A method according to claim7, wherein the step D3) further includes the step of determining theaverage value of the squares of the differences between each pair ofcorresponding pixel values in the blocks provided by the respectivefirst and second block conversion means.